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Workshop  Objectives 


•  Define  a  research  agenda  for  the  next  decade  and  beyond 

•  Survey  research  needs  for  theories,  methodologies,  algorithms  in 

•  System  theoretic  issues  for  deterministic  and  stochastic  systems 

•  Linear  and  nonlinear  robust  control 

•  System  identification 

•  Robust  adaptive  control 

•  Fault  tolerant  and  reconfigurable  systems 

•  Hybrid  control 

•  Hierarchical,  decentralized,  and  distributed  large  scale  systems 

•  Intelligent  systems 

•  Chaotic  and  complex  systems  ( a  la  Santa  Fe  Institute  ...) 

•  Novel  concepts 


Optimal  Control  Theory 


•  Maximum  Principle  of  Pontryagin  and  Bellman’s  Dynamic 
Programming  are  the  foundations  of  optimal  control  theory 

•  why  no  new  powerful  theorems  in  the  last  35  years? 

•  Neurodynamic  programming  uses  iterative  methods  to  approximate 
cost-to-go  function  and  store  it  in  a  neural  network 

•  One  could  also  use  repetitive  numerical  solutions  of  the  two-point- 
boundary-value  problem  to  train  neural  network  for  approximate 
implementation  of  optimal  nonlinear  feedback  control  law 

•  Combination  of  genetic  algorithms  with  classical  algorithms  to  handle 
issues  of  locally  optimal  solutions 

•  Efficient  and  reliable  numerical  methods  for  approximating  solution  of 

Hamilton-Jacobi-Bellman  equation  would  lead  to  major  advances  in 
several  domains  ^ 

•  Are  there  new  advances  in  dynamic  team  and  dynamic  game  theory 

and  algorithms?  .  ,  ;  , 


Estimation  Theory 


I 


•  Classical  Kalman  filters  and  (suboptimal)  extended  Kalman  filters 
remain  the  workhorses  of  applied  estimation 

•  Blending  of  hypothesis-testing  concepts  and  filtering  algorithms  can 
lead  to  significant  performance  payoffs 

•  driven  by  multi-sensor  multi-object  tracking  problems  that  include 
changes  in  maneuvers,  temporary  occlusion  of  target,  motion 
constraints  etc. 

•  performance  vs.  combinatorial  complexity  tradeoffs  are  required 

•  Distributed/decentralized  estimation  problems  are  much  more  difficult 
than  their  centralized  counterparts 

•  example:  decentralized  team-theoretic  hypotheses-testing 
problems 
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Robust  Muitivariabie  Controi 


•  Feedback  imperative  to 

•  stabilize  open-loop  unstable  plants  and/or 

•  deliver  superior  performance  in  the  presence  of  plant  and  exogenous 
signal  uncertainties 

•  Robust  control  issues:  in  the  presence  of  parametric  and  unmodeled  dynamic 
errors,  guarantee 

•  stability-robustness 

•  performance-robustness 


Robust  Linear  Control  Designs 


Model  Errors 


Disturbances 
d(s) 


Commands 


Structured  errors:  known  bounds  on  uncertain  parameters 
Unstructured  errors:  unmodeled  dynamics  (possibly  Infinite-dimensional) 
bounded  in  the  frequency  domain 

For  all  legal  structured  and  unstructured  uncertainty  must  guarantee 

•  stability-robustness 

•  performance-robustness  to  posed  specifications  (command-following 
disturbance-rejection,  insensitivity  to  sensor  noise) 


Mixed  /X  -synthesis  Setup 
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Generalized  plant 
Dynamic  compensator 


H-oo  Problem  in  DK  and  DGK  Iteration 


Directions  in  Linear  Robust  Controi 


•  Mixed  |i-synthesis  methodology  well  established 

•  computationally  intensive  for  several  uncertain  parameters, 
leading  to  compensators  of  increasing  dimensionality 

•  Integration  with  adaptive  control  to  reduce  parameter  uncertainty 
bounds? 

•  Alternatives  for  robust  synthesis  (must  be  valid  for  both  structured 
and  unstructured  uncertainty) 

•  view  parameters  as  random  variables? 

•  other? 
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Noniinear  Feedback  Controi  Methods 


•  Research  goal  is  to  design  for  stability  and  performance  robustness 

•  Common  present  methodologies 

•  Lyapunov-based  designs 

•  Gain-scheduling 

•  State-feedback  linearization 

•  Sliding-mode  controllers 

•  Backstepping  methods 

•  Do  we  need  a  new  paradigm? _ _ _ 
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Gain-Scheduling 


•  To  obtain  non-conservative  stability-robustness  results,  one  needs  to  routinely 
solve  Hamilton-Jacobi-Bellman  partial  differential  equation  inequalities 
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System  Identification 


•  Open-loop  vs.  closed-loop 

•  role  of  excitation  signals 

•  Classical  methods  have  difficulty  when  there  are  several  poles  and 
zeroes  near  the  jco-axis,  especially  in  MIMO  settings 

•  Identification  for  robust  control 

•  Identification  in  behavioral  modeling  setting 
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[dynamic  SYSTEM] 
||  (PLANT)  I 


Classical  adaptive  MIMO  designs  ackward.  Stability-robustness  to 
unmodeled  dynamics  still  a  big  problemidentification  for  robust  control 
Sequential  identification/robust-controller  resynthesis? 

Robustified  Multiple-Model  Adaptive  Control  (MMAC)  methods 
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Classical  Adaptive  Control 


•  Traditional  methods  (MRAC,  STURE, ...)  developed  for  plant 
parameter  estimation;  do  not  take  into  account  unstructured 
uncertainty 

•  focus  on  command-following,  not  disturbance-rejection 

•  difficult  to  generalize  to  MIMO  systems 

•  stability-robustness  not  guaranteed,  ad-hoc  fixes  still  do  not  lead 
to  global  stability  guarantees 

•  unmeasurable  plant  disturbances  and  sensor  noise  can  create 
stability  and  performance  problems 

•  Benefits  of  adaptive  control  should  be  quantified 

•  must  quantify  tradeoffs  of  using  persistent  excitation  that  increase 
output  disturbances  vs.  longer-term  performance  improvements 
due  to  learning 

•  Operation  in  closed-loop  not  well  understood 


Sequential  Identification/Compensator 
_ Redesign _ 


♦  Continuous  readjustment  of  compensator  parameters  seems  to  be 
wrong  way  to  go 

♦  Sequential  closed-loop  identification/compensator  redesign  may  be 
superior  engineering  approach 

•  advances  in  identification  for  robust  control 

•  needs  rethinking  of  proper  persistent  excitation  signals 

•  should  take  advantage  of  major  advances  in  the  design  of  robust 
linear  controllers  (mixed  p-synthesis,...) 

•  open  closed-loop  bandwidth  slowly,  monitor  for  instabilities  so  as 
to  revert  to  prior  stabilizing  compensator 

•  should  improve  performance  degradation  caused  by  “detuning”  of 
fixed  compensators  because  of  large  parameter  errors 


Multiple  Model  Adaptive  Estimation  (MMAE) 


H 

1 

H 

1 

N 

^T#N; 

Si(t)  ^ 

S2(t)  Z 

m)  Tt 

urn 


Posterior 
[  Probability 
I  Evaluator 


Residual 
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Posterior  hypotheses 
probabilities 


If  true  system  is  included  in  bank  of  Kalman  filters,  it  is  identified  with 
probability  one.  Otherwise,  we  identify  nearest  “probabilistic  neighbor" 


Posterior  hypotheses 
probabilities 


Applied  control  is  the  probabilistic  average  of  “matched  LQG  controls” 
Other  variants  possible,  e.g.  apply  control  with  the  largest  current  posterior 
probability  (switching...)  _ 
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Fault-Tolerant  Control 


•  Detect  soft/hard  failures  in  sensors  and  actuators,  abrupt  dynamic  changes 

•  Impact  of  parametric  and  dynamic  errors  on  FDI  algorithms,  e.g.  GLR  .. 

•  Controller  reconfiguration  algorithms 

•  Role  of  hybrid  system  theory  (mode  switching...) 

•  Role  of  “intelligent”  supervisory  methods 

•  Robust  stability  and  performance  guarantees 


Hybrid  Control 

Discrete-state  system _ 


!  I 


1 
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Continuous-time  system 


•  Architectures  involving  interactions  between  a  finite-state  event-driven 
system  and  a  continuous-state  continuous-time  system 

•  Discrete  level  can  establish  different  modes  of  operation  for  feedback  system 
based  upon  feedback  of  behavior  of  continuous  variables 


Issues  in  Hybrid  Control 

•  Deals  with  switched  dynamical  sytems,  software  enabled  control,  and 
multimodal  control 

•  Impact  on  several  disciplines  and  applications,  e.g. 

•  fault-tolerant  reconfigurable  control 

•  hierarchical  systems 

•  Formal  modeling  leads  to  some  qualitative  properties  (reachability,...) 

•  difficult  to  establish  existence,  uniqueness  and  sensitivity  of 
solutions 

•  some  verification  software  is  available 

•  Design-directed  theories,  methodologies,  software  are  at  their 
infancy.  They  seem  to  require  advances  in 

•  optimal  control  theory 

•  dynamic  team  theory 

•  dynamic  game  theory 

•  Complexity  and  combinatorics  are  main  obstacles 
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Decentralized  Control 


•  Weak  dynamic  coupling 

•  Multiple  time-scales 

•  Stability  and  performance  robustness  guarantees? 
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Issues  in  Decentralized  Control 


•  Non-classical  information  patterns  “fighf  with  classical  optimization 

•  signaling  strategies 

•  second-guessing  strategies 

•  cost  of  communications  and  communications  delays 

•  “easy”  convex  centralized  problems  transform  into  “nasty” 
nonconvex  combinatorial  ones  in  decentralized  setting 

•  WHO  should  know  WHAT,  WHEN,  WHERE  and  WHY 

•  Performance  impact  of  enhanced  information  patterns  in  teams 

•  Cooperative  multiperson  dynamic  game  theory  can  be  exploited 

•  Nash  strategies 

•  Stackelberg  (leader-follower)  strategies 

•  Can  we  isolate  superior  alternative  hierarchical  and  spatial 
decompositions  and  architectures? 

•  Is  there  a  decentralized  “maximum  principle?” 


Intelligent  Systems 


•  Fuzzy  systems 

•  fuzzy  feedback  control 

•  fuzzy  supervisory  layers  (fuzzy  hybrid?) 

•  uncertainty  modeling? 

•  Neural  networks 

•  training  and  learning 

•  specific  means  of  approximating  nonlinear  static  maps 

•  Genetic/evolutionary  algorithms 

•  minimal  assumptions  on  problem  structure 

•  used  for  bypassing  local  optima 

•  require  extensive  number  of  iterations  for  convergence 

•  hard  convergence  and  rate  of  convergence  proofs 

•  All  three  have  attracted  a  lot  of  attention  in  computer  science 
community.  Normative  vs.  empirical  approaches. 


Chaotic  and  Complex  Systems 


•  Chaotic  systems:  consequence  of  solution  properties  for  classes  of 
systems  of  ordinary  differential  equations 

•  analysis  by  simulation  studies 

•  Complex  systems:  macroscopic  behavior  of  systems  that  contain  a 
very  large  number  of  interacting  simple  agents  with  specific  survival, 
reproductive,  etc.  characteristics 

•  artificial  life:  studies  involving  large-scale  simulations  providing 
trends  of  species  survival,  migration, ... 

•  What  is  role  of  systems  and  control  theory  (apart  from  simulations 
with  sexy  graphics?) 

•  Recent  research  on  “Highly  Optimized  Tolerance”  by  J.C.  Doyle 


Concluding  Remarks 


•  Our  heritage  is  fifty  years  of  exciting  and  important  developments  in 
theory,  design  methodologies  and  algorithms 

•  There  exist  numerous  opportunities  for  relevant  basic  theoretical  and 
methodological  research 

•  adaptive,  multimodal,  complex  systems 

•  my  favorite:  software  that  routinely  solves  Hamilton-Jacobi- 
Bellman  partial  differential  equations 

•  Expanding  applications  that  benefit  from  theoretical  developments 

•  Should  always  be  on  lookout  for  significant  engineering/scientific/ 
socioeconomic  applications  that  suggest  novel  theory  development 
and  extensions 

•  Expect  continuation  of  “clashes”  between  ad-hoc  approaches  from 
the  computer  science  community  and  the  quantitative  model-based 
methodologies  of  our  discipline 

•  we  shall  overcome! 
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OF  ADAPTIVE  CONTROL 

»  Changing  experimental 
conditions  accurate  but  inexact 
models 

» Impractical  control  objectives 
»  Transient  instability 

VINNICOMBE  METRIC 

ITERATIVE  CONTROL  AND 
IDENTIFICATION 

ADDRESSING  THE  FUNDAMENTAL 
PROBLEMS 

MULTIPLE  MODEL  ADAPTIVE 
CONTROL  AND  VINNICOMBE 
METRIC 


CONCLUSIONS 


A  Result  with  Practical 
Difficulties 


•  Let  P  be  unknown  linear  system. 

•  Let  P  have  known  degree  and  be 
minimum  phase. 

•  [Statements  regarding  operation  of  an 
adaptive  controller.] 

•  Then  as ? all  signals  in  the 
adaptive  loop  remain  bounded,  the 
controller  converges  and  the 
performance  Index  is  minimized  (i.e. 
asymptotically,  all  is  right  with  the 
world). 

•  This  result  sidesteps  three  problems 
which  arise  in  adaptive  control, 
including  adaptive  control  of  nonlinear, 
distributed  etc  systems. 


Problem  Of  Changing  Experimental  Conditions  with 
Accurate  but  Inexact  Models 


•  Arbitrarily  small  open-loop  modelling 
errors  can  lead  to  arbitrarily  bad  closed- 
loop  performance,  when  a  nice 
controller  is  designed  for  the  model  and 
connected  to  the  real  plant. 

•  Large  open-loop  modelling  errors  may 
give  small  closed-loop  errors. 

Example:  P^  =  Po+  high  frequency 
resonance,  C  stabilizes  Pq  with  closed- 
loop  bandwidth  excluding  the 
resonance. 

•  Problem  is  that  closeness  of  models  is 
evaluated  with  particular  experimental 
conditions. 
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(^  +  l)(0.1i-  +  l) 


Different 
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Problem  Of  Changing  Experimental  Conditions  with 


Accurate  but  inexact  Modeis 


I 


•  Insertion,  removal  or  change  of  a 
controller  is  a  change  of  experimental 
conditions. 


•  Qualitatative  conclusion:  controller 
changes  should  be  small. 

•  Question:  How  can  we  measure 
smallness? 


5 


Problem  of  Impractical  Control 

Objective 

•  Some  non-adaptive  control  problems, 
while  theoretically  soluble,  have 
impractical  solutions. 

Example:  LQG  problem  yielding  e 
degrees  phase  margin  for  the  loop. 


•  If  one  tried  an  adaptive  control 

solution,  not  knowing  the  plant  initially, 
there  would  be  trouble  (large  transient 
signals,  failure  to  converge  etc.) 


•  Question:  How  can  we  arrange  that 
adaptive  control  algorithms  flag  over- 
ambitious  control  objectives,  and  back 
off  in  seeking  to  achieve  them? 
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Problem  of  Transient  Instability 


•  “All  signals  in  the  adaptive  closed-loop  remain 
bounded”  permits  10^^  arnpg  motor  current. 

•  A  property  of  one  famous  adaptive  control 
convergence  result  is  the  following: 

Let  (^0  denote  the  vector  of  initial  conditions  and 
Z(t),  the  vector  of  all  interesting  signals  at  time  t. 

For  arbitrary  N>0,  there  exists  with  ||^ol|  <  1 


and  >  0  such  that  ||z(ri)||  >  N 


\  ~\y  . 

rKi  {imn  ffy 

w,  y:! 

•  Suppose  that  at  f  =  0,  we  know  how  to 
connect  a  stabilizing  controller.  How  can  we 
adapt  it  -  not  knowing  the  plant  -  to  retain 
frozen  closed-loop  stability? 
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Generalized  Stability 
Marain 


T{P,  C) 


UJ 


T{P,C) 


p{i+cpy^c  p{i+cpy^ 
{i+cpyk  {i+cpy^ 


{i+cpy\c  I] 


!  •  Define  generalized  stability  margin  by 

bp,c  =  01i:‘  if  C)  is  stable 

=0  otherwise 

-  small  if  close  to  instability, 

j 

(1  +  PC  close  to  zero) 

-  small  if  closed  loop  bandwidth 

exceeds  open  loop  [C(1  +  PC)'^  large] 
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Generalized  Stability 


•  Measure  of  difficulty  of  controlling  a 

Sup 

particular  plant  P  is  b^p^iP)-  bp  c 

-  Nice  formula  available,  showing  that  right 
half  plane  poles  or  zeros  make  b^ptiP)  smaller, 

i.e.  lTopt{P)\^  bigger 

-  nonlinear  version  available 
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t 
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Vinnicombe  Metric  and  Link  to 


*  Introduce  distance  measure  between  two  plants 
For  scalar  plants, 


t 


'(^1.^2) 


max] 
=  0) 


p?  - 


I  I 

(l  +  |F2p)2(l  +  iPih2 


(provided  a  winding  number  condition  is  satisfied),  and 
^vCPj,  P2)  ^  1  otherwise. 

-  nonlinear  and  multivariable  version  available. 


•  Suppose  (Pi,C)  is  stable.  Then 
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Vinnicombe  Metric  and  Link  to 

Stability  Margin 


•  Suppose  iP,C])  is  stable.  Then 
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Consequences  of the 
^^^etric^Mai^ii^onnection 

I  •  Pj  hard  to  control  boptiPi)  is  small 

«  b{Pi,C)  is  even  smaller  for  some 
stabilizing  C 

<=>  little  scope  to  change  C  without 
risking  instabiiity 

i  little  scope  for  modelling  errors 

in  Pi,  or  satisfactory  behaviour 
with  C  shouid  Pi  drift 

j  •  Small  changes  in  C  (=  safe  changes  in  C)  require 

knowledge  of  bp  c  for  their  determination. 

•  Suppose  q  stabilizes  Pi  and  C2  replaces  Q  with 

8y(Ci,C2)<bpci 

C2  may  give  less  robust  design  than  Q  : 

sin“^ Zjp (7  > sin”^ fjp Q  -5y{Ci,C2) 

and  equality  is  possible. 
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Consequences  of  the  |; 
etric/MaraIn  Connection  i 


5v(Ci,C2)<|lr(p,Ci)-r(p,C2)|L< 


8ACi,(h) 

h,Ci  h,C2 


8^{C^,C2)4T{P,C^)-T{PX2)t  ^ 


lT{PA)t  8y{Cl,C2) 

1-  7’(P,Cj)  ^5y(Cj,C2) 
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Iterative  Control  and  Identification  i 

^^^^^^^Algorith^^Rjameworj^^^^^^jj 

1  Assume  controller  Ci  is  connected  to  real  plant  P  and 

I'-l 

gives  stable  closed  loop.  ■  t 

li 

2  Identify  a  model  Pi  of  P  using  closed -loop  data. 

3  Using  R  redesign  controller  to  obtain  Cj_,..|  -  ensuring  Cj^..||i 

is  not  too  different  to  Cj.  w 

4  Check  that  (Pj,C|+J  behaves  like  (P,C|+i).  If  so,  update  i  f] 

and  go  back  to  step  3.  If  not,  go  to  step  5.  1 1 

5  Reidentify  P  to  obtain  a  new  model  Pj+-|,  connected  to  a  | ; 

controller  I 

•  Repeat  cycle  until  desired  performance  is  attained  or  is  ■ 
seen  to  be  unattainable.  * 

I 
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Iterative  Control  and  Identification 
Algorithm  Framework 


'  •  For  plants  with  only  unstable  poles  at  origin,  and  with 
'  Cj  obtained  by  IMC  design  (with  iterative  closed  -  loop 
bandwidth  broadening),  windsurfer  approach  to 
‘  adaptive  control  results. 
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Cautious  Contoller  Adjustment: 
What  Is  Acceptable? 


•Assumption;  the  statement  that  “Pj  is  a  good  model  of  P 

using  closed -loop  data  with  controller  C," 

means 

T{P,Ci)  =  T(PjA) 


•  Hence 


Hence  we  should  select  Q+i  such  that 

J  ^ 
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Cautious  Contoller  Adjustment: 
What  is  Acceptable? 


Assumption :  for  unknown  P,  the  design  goal  is  to  obtain 
stabilizing  C  to  minimize  J{P,  C). 

Let  P  have  right  coprime  description  ND'^  and  C,  right 


coprime  description  UV  \ 

c  =  {C{Q) :  C(Q)  =  iu-  DQ)  {V  +  NQY^] 
for  Q  stable  delineates  set  of  stabilizing  compensators 


i  •  Assumption:  7(p^.,C(0)forC€  C  depends  on  g  in  a 
convex  manner. 

In  H2  and  dependence  is  convex  ] 
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Cautious  Contoller  Adjustment: 
What  is  Acceptable? 


W  5^(Ci,CM)<kbp.  choose  Q+i -C*+i 


Else  look  at  set  for  a  e[0,l] 

c(aQ*]  =  {u-aDQ*){y  +  aNQ*) 

a  =  0:C=Ci  a  =  l:  C=C-+i 
Choose  a  e  (0, 1)  with 

Sv(Ci,c(aQ*))  =  kbp,(., 

Q+i  =  c(a2*) 

and 

j(Pj,q+-i)<j(Pj,Ci) 

(Safety  and  performance  improvement) 
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Addressing  the  Fundamental 

Problems 


Changing  experimental  conditions  given  accurate  but 
inexact  models 

-  Limit  5y(C|,C|+-|)  using  T(Pj,Cj) 

-Rely  on  ||T(P,C|)L£  T(Pj,Ci)| 


Impractical  control  objectives 

Possibility  1 :  Identification  may  not  be  sufficiently 

reliable  to  conclude  T(Pj,Cj)  =  T(P,Cj) 

r2  =  0 

-  Suppose  closed  loop  bandwidth  is  (Oq 

-  Suppose  estimate  PC(1+PC)  ^  at  lOr^o,  SNR  =  1 

-  P(1  +  PC)~^  at  10a)o  estimate  will  have  SNR  =  1 

-  If  P  has  resonance  at  lOo^o  not  captured  in  Pj,  ||T(P ,C)L 

could  be  due  to  P(1  +  PCr'‘  at  lOroo  and  misestimated.  ^2 


Addressing  the  Fundamental 

Problems 


•  Possibility  2 :  T(Pj,Cj)  may  be  very  large,  so  scope 
'  for  adjusting  Cj  is  negligible. 

i 

1  •  Transient  instability 

'1 

] 

‘J 

!  -  Assume  stabilizing  compensator  is 

known  for  the  unknown  plant. 

1 

I 

Then  transient  instability  is 
!  avoided  (so  long  as  controllers  are  not 
switched  too  fast). 


THREE  FUNDAMENTAL  PROBLEMS 
OF  ADAPTIVE  CONTROL 


VINNICOMBE  METRIC 

ITERATIVE  CONTROL  AND 
IDENTIFICATION 

ADDRESSING  THE  FUNDAMENTAL 
PROBLEMS 

MULTIPLE  MODEL  ADAPTIVE 
CONTROL  AND  VINNICOMBE 
METRIC 


CONCLUSIONS 


Multiple  Model  Adaptive 
Control  and  Vinnicombe  Metric 


0.3  <  A  <  3 


Consider  a  set  of  plants 

W  =  ^ 0.3<1<3 

is-2)(s  +  l) 

or  an  even  bigger  set 

n  =  {P :  P  =  P^is)[l  +  A(5)]with||A(;t9)t  <  d} 


'  •  Multiple  model  adaptive  control  assumes 
there  exist  N  and  \  < ...  <  such  that 

'  -  a  nice  controller  C;^  exists  for 

-  any  plant  in  11  is  controlled  nicely  by 
some 


•  Multiple  model  control  provides  an  algorithm 
to  switch  among  the  Safety  is  an  issue. 


24a 


Multiple  Model  Adaptive 
Control  and  Vinnicombe  Metric 


•  Answer  for  P;e(s) 

i —  A.-  — ^ 


0.3 


A... 


•  All  plants  in  A,-  obey 


<0.1 


3 


This  ensures 


T{P,^,C^)  -  T(P,„C^.)\l  <  o.i||r(P^.,c,, 
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OUTLINE 


THREE  FUNDAMENTAL  PROBLEMS 
OF  ADAPTIVE  CONTROL 

VINNICOMBE  METRIC 

ITERATIVE  CONTROL  AND 
IDENTIFICATION 

ADDRESSING  THE  FUNDAMENTAL 
PROBLEMS 

MULTIPLE  MODEL  ADAPTIVE 
CONTROL  AND  VINNICOMBE 
METRIC 

CONCLUSIONS 

»  Main  messages 

»  Shortcomings  and  future 
possibilities 


Main  Messages 


•  Adaptive  control  should  have  modest  steps 
only  in  the  controller. 

•  The  Vinnicombe  metric  plus  closed-loop 
measurements  Indicate  what  Is  modest. 

•  A  desired  control  objective  may  be  impractical; 
good  algorithms  need  to  flag  this  and  adjust 
their  goal.  Some  do  this. 

•  Transient  instability  should  be  avoided,  and 
can  be  avoided. 
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Shortcomings  and  Future 


Possibilities 


The  Vinnicombe  metric  is  a  single  number.  It 
would  be  better  to  look  at  the  whole  imaginary 
axis 

There  is  no  protection  against  step  changes  in 
the  plant  (Hypothesis  testing  rather  than 
parameter  estimates  should  be  used  if  possible 
to  identify  change) 

There  is  a  lack  of  methodology  for  optimally 
using  closed-loop  data  obtained  for  one  plant 
and  more  than  one  controller  in  identifying  that 
plant 

The  ideas  look  fruitful  for  nonlinear  problems. 
Much  has  already  been  done  on  the 
Vinnicombe  metric  /  robust  stability  question. 


IdGntiflcation  of  ComplGX  SystGins 


Munther  A.  Dahleh 


Department  of  Electrical  Engineering  and  Computer  Science 
Laboratory  for  Information  and  Decision  Systems 

MIT 


Motivation 


Robust  Control  Theory:  Major  development  in  the  past  20  years. 
Design  tools 

-  Start  with  the  nominal  process,  a  description  of  the  unmodeled  dy¬ 
namics  and  exogenous  inputs. 

-  Provide  performance  tradeoffs. 

An  example  of  an  1/ 0  description  of  a  process. 

y  =  Gu 5Gu  -\- Vi/ w . 

System  identification  has  not  had  a  parallel  development. 

Existing  theory  did  not  address  control-oriented  modeling. 


Objective 


•  Develop  a  paradigm  for  identification  of  complex  systems  using  limited- 
complexity  models. 

•  Motivation:  consistency  with  existing  robust  control  paradigms. 

•  Main  considerations 

-  Explicit  under  modeling. 

—  Error  measures. 

-  Noise  (deterministic  or  stochastic). 

•  Standard  issues 

—  Experiment  Design. 

-  Algorithms  and  Computations. 

—  Error  bounds. 

—  Complexity. 

•  Reference:  S.R.  Venkatesh.  Identification  for  complex  systems.  Ph.D. 
Thesis  #  LIDS-TH-2394,  MIT,  Cambridge,  MA. 


Pohnst  Control 


•  Interconnection: 

-  G:  Nominal  system. 

—  A:  Describes  plant  uncertainty. 

-  K  stabilizing  feedback  controller. 

—  w:  exogenous  input. 

—  z:  regulated  output. 

•  Uncertainty: 

—  Uncertainty  description  is  linear  fractional. 

—  Example:  Additive  or  multiplicative  uncertainty. 

_ A  is  an  unknown  system,  with  bounded  induced  norm  (^oc-,  ^i)- 


(MPE)  System  Identification 


Minimum  Prediction  Error  Paradigm. 

y  = 

-  u;  is  a  stochastic  Process  (generally  white  noise)  . 

-  A  parametrized  set  of  models. 

-  Actual  process  is  in  the  parametrization. 

-  Estimate  the  parameters  from  Data  by  minimizing  the  “prediction 
error” . 

Positive  results. 

-  Consistency,  if  actual  process  is  in  the  model  parametrization. 

-  Asymptotic  distribution  of  the  parameters. 

-  Sample  complexity  issues. 

Limitations 

-  Stationarity  assumptions  on  the  noise. 

-  Results  are  asymptotic. 

-  Under  modeling  is  not  accounted  for  a  priori.  Equivalently,  noise  is 
the  only  source  of  mismatch. 

-  Difficult  to  integrate  with  existing  control  paradigms. 


Spt-Membership  System  Identification 


Prior  Information 

—  The  Process: 

y  =  G{9q)u  +  A(u)  +  w 

—  w:—  output  noise,  ly  G  W. 

—  A:=  Plant  uncertainty,  A  G  A. 

—  model  parametrization  M{9)  =  {G{B)  +  A,  A  G  A}  C  X,  X:= 
model  set. 

—  0  €  ©  G  IR” 

—  u:=  Input,  u  Q.U  (|lul|oc  ^  I)- 
Positive  issues 

—  Consistent  with  robust  control. 

—  Explicit  under  modeling. 

Negative  issues 

—  No  clear  separation  between  the  finite  models  and  the  unmodeled. 

—  VV  =  Bloc,  is  a  conservative  model  for  noise. 


Problem  Description 


•  Process  class,  T,  contains  stable  LTI  systems, 

-  Set  is  complex;  not  approximable  by  a  finite  dimensional  set. 

-  equipped  with  a  metric. 

—  Real  process  is  in  this  set. 

•  Model  parametrization  Q 

—  Finite  dimensional  set  (subspace). 

-GcT 

•  Noise 

-  Stochastic  or  deterministic. 

-  Deterministic:  set  is  not  too  rich  or  too  constrained. 

—  Set  description  of  white  noise. 

•  Data:  finite  observations  of  the  input  and  output. 


y  =  Tqu  +  w 


Prnhipm  Formulation 

•  Output  equation: 

y  =  Tqu  +  w  ToGT,'u;e>V 

•  Define  Gq  as 

Go  =  argmm|lTo-Gll 

•  Prior:  There  exists  a  7  such  that  dist(To,  Q)<1- 

•  Problem:  Does  there  exists  an  input  w,  and  an  algorithm  Gn  so  that 

1.  limn->oo  SUPr^,^  H^o  -  <5nl!  =  0, 

2.  Time  complexity  of  convergence  is  polynomial. 


G 


Comments 


•  Choice  of  T 

—  Operator  spaces  {Hooi  h)- 

-  The  space  (.2  more  convenient  mathematically. 

-  Hardy-Sobolev  space  Hilbert  space  with  inner  product 

<  f^9  >n=<  f,9>  +  <  f'^9'  > 

-  The  following  inequality  holds: 

IITlloo  <  llTlli  <  c\\T\\n 

-  Robust  control  can  deal  with  Ti. 

•  Prior  structure  is  additive.  Need  more  complicated  structures. 

•  Outcome  of  identification: 

—  An  estimate  Gn- 

-  A  (parametric)  estimate  of  the  error  |lGo  -  G„l|,  Dn- 

-  An  estimate  %  of  the  iminodeled  part  %  <  7. 

-  Additional  structure  on  the  unmodeled  part,  A  =  0-^. 

—  The  model 

{Gn  +  A  1  F  -  G„|l  <Dn,A€  A,  II  All  <  7n} 


r>ftsr.ription  of  the  Unmodeled 


•  Go  is  the  best  approximation  of  To  in  T  -  U. 

•  Express  To  as 

To  =  Go  +  ^0 

Ao  G  Q^- 

•  Example:  Q  is  the  set  of  FIR  of  order  M,  then 

f  arbitrary}. 

•  Example:  Q  =  {Oil  —  ciX,  0  G  IR},  then 

6!-  =  {A|  E(H-fcV'5t  =  0  } 

fc=0 


•  If  ^  is  a  subspace,  then  A  is  nice  set! 


I  Diameter  of  Uncertainty 

> 

I 

]  •  Set  of  unfalsified  models: 

!  S„(y,«)  =  {G  €  aiy(n)  =  G«(n)+An(n)+»(n),  A  6  A,  1|A||  <  7,  w  £  W} 

i 

!  .  • 

•  DistmctBr  of  imcGrtainty  3.t  tiniG  ti. 

I  DniT,  Q,  W,  u)  =  sup  sup  diam(5„) 

TeT^^y\; 

] 

^  •  Asymptotic  diameter  of  uncertainty  D  =  limn^{x  Dn- 

I  •  Problem:  Does  Dn  converge  to  zero  for  some  input  u  in  polynomial  time? 


Results 


Exact  modeling 

-  Bounded-but-unknown  noise. 

-  Low  correlated  noise  models. 

-  Bridge  the  gap  between  stochastic  and  deterministic  formulations. 

Under  modeling 

-  FIR  model  structures 

-  General  finite  dimensional  subspaces  of  finite-dimensional  systems. 

Identification  in  practice. 

Connection  to  robust  control. 


Correlation  Methods:  Exact  Modeling 


•  Problem  Set-up: 

-  y  =  GqU  -h  w 

-  Go  E  ^  is  the  subspace  of  FIR  models  of  order  M. 

•  The  estimate  is  computed  as  follows: 

ryu{r)  =  GNru{T).  t 

•  When  does  G.v  converge  to  Go? 

•  Input  is  persistently  exciting  of  order  M. 

•  Noise: 

-  quasi-stationary,  uncorrelated  with  u  (e.g.,  white  noise). 

-  Set-valued  with  Correlation  Constraints:  (7^  < 

W,v  =  {d  €  IR-'^  1  ^  E  1^?  ('^)1  ^ 

T=1 

-  Set-valued  with  Periodogram  Constraints: 

W?:-  =  Id  e  IR"|  .7  E  4e“  <  W{w),  w  €  10.27r]| 

\  iog  N  k=o  ) 

-  Both  W.v,  Wv  contain  white  noise  with  high  probability. 

•  Convergence  is  unitorm  with  polynomial  sample  complexity. 

•  Does  not  converge  if  noise  is  only  "bounded-but-imknown  " . 


Under  Modeling:  Basic  Idea 


Two-step  algorithm 

1.  Filter  the  data  to  eliminate  the  unmodeled  part. 

2.  Use  previous  correlation  method. 

T  is  constructed  from  the  annihilators  of  A. 

What  about  the  input? 


Input  Design:  Motivation 


•  ^  is  the  subspace  of  FIR  models  of  order  0  (constants). 

.  A  =  (0,  fi,  h,  ■  ■  .)• 

•  y{k)  =  gou{k)  +  /,«(*:  -  J)  +  Mk) 

•  Tradeoff 

-  If  -u;  =  0,  then  the  best  u  is  the  unit  pulse. 

-  If  u;  0,  then  u  needs  to  be  longer. 

•  Periodic  inputs  will  not  work: 

y(k)  =  (90  +  /i  +  •  •  +  (/i  +  +  ■  •  X''  -  1)  +  ■  ■  ■  +  “■'(*;) 

•  Obtain  first  order  correlations  with  u 

»-i:;:=!;or„''(0)  +  i;/irf(i)+4.(0) 

vf=l 


Conditions  on  the  input  (Robust  Input) 

Hi) 


1.  supkka- 


ril(O) 


(0) 


— )■  zero. 


'(0) 


2. 


zero. 


Input  Design 


Not  robust  inputs 

—  Periodic 

-  Standard  Chirp  u{t)  =  exp{iaf),  a  G  K 

ku  W 


max 

\<i<N 


^(0) 


>1  —  6  for  any  e  and  some  N 


—  PBRS:  For  large  period  m,  correlations  up  to  m  can  be  made  small. 
Not  sufficient. 

•  Robust  Inputs 

-  High  order  chirp  u(i)  =  exp(iQ;t^),  a  =  7r(l  +  \/5): 


max 

i<j<Ar 


rm 


<C 


log(iV) 


-  i.i.d.  Bernoulli  process  with  zero  mean  and  variance  1 


V  I  max.rf  (i)  >  cti  <  nexp(-n/(Q!)) 
I  l<i<A'  J 


-  For  both  signals 


rL(0) 


ri'(O) 


zero 


FTB  Models 


•  Assumptions: 

-  T  is  or  H. 

-  g  IS  the  set  of  FIR  systems  of  order  M. 

-  Noise  is  in  W. 

-  Prior  I:  dist(To,a)  is  bounded  by  7. 

-  Input:  High  order  chirp. 

•  Results 

-  The  least  squares  estimator  converges  to  Gq: 

10^(71) 

sup  sup  llG'ri  -  Goll  <  (C'i7  +  C2M) 
T6i,„eVV 

-  The  estimator  is  untuned 

-  Complexity  is  polynomial. 


Fyample:  One-Pole  Model 


•  e  =  {ra. « s  ®-}- 

•  T-h  (for  simplicity). 

•  A  =  {(A  —  o)/(A),  /  €  ^2}- 

•  Equivalent  output  equation: 

1 

yijc)  =  9u{k)  +  {X  — ^  ~  1  —  aA^ 

•  Comments 

-  There  is  no  direct  separation  between  the  model  and  the  unmodeled 
dynamics. 

-  Least  squares  estimator  will  not  converge.  The  unmodeled  part  af¬ 
fects  r„'(0). 

-  Need  to  exploit  the  structure  of  the  unmodeled. 

•  A  two-step  approach 

-  Pre-filter  the  data  using  the  non-causal  filter  a  This  corresponds 
to  multiplying  the  data  by 

(I  a  a-  ...  \ 

1  ,  _  ^N-l 


-  Apply  a  least  squares  estimator  to  the  filtered  Data. 

-  Can  interpret  as  a  weighted  LS. 


Of>Tieral  Result 


•  Assumptions 

-r=n. 

-  Q  k  finite  dimensional  subspace  of  stable  systems,  with  dimension 

M. 

-  Noise  is  in  W. 

-  Prior  I:  dist(To,^)  is  bounded  by  7. 

-  Input;  High  order  chirp. 

•  Results 


-  There  exists  a  filter  such  that  the  two-step  algorithm  converges  to 


Go 


sup  sup  !!G„  -  Goll  <  (C'i7 
T€l,,g>V 


log(^'0 


-  The  estimator  is  untuned 

—  Complexity  is  polynomial. 


MPE  Revisited 


•  If  ty  is  WN,  then  it  can  be  shown  that  for  a  fixed  To  G  T,  the  LS  estimator 

satisfies:  _  i 

Urn  G.y  =  G  =  minlKTo  -  G)$s||2  w.p.l 

N-^oo  *  Gey 

where  is  the  spectral  density  function  of  u. 

•  Comments: 

—  Convergence  is  only  pointwise  (w.p.l). 

-  There  is  no  uniform  convergence  rate. 

—  Results  are  asymptotic. 

—  Results  require  quasi-stationarity. 

-  G^Go  unless  the  input  is  robust. 

-  No  robust  inputs  are  constructed. 

-  Theory  addresses  £2  error  bounds. 

•  Contrast  with  previous  results: 

—  Convergence  is  uniform  (w.p.l  if  w  is  W^N). 

—  A  uniform  convergence  rate  is  derived. 

—  Estimates  are  based  on  finite  data. 

-  No  stationarity  assumptions. 

-  Robust  inputs  are  constructed. 

-  Theory  addresses  more  general  error  bounds. 


TH^ntification  In  Practice 


•  The  set  of  unfalsified  models,  S'jv  is  convex. 

-  Define  Zy,{t)  =  y{t)  —  Gu{t)  —  w{t)  for  i  =  0,  . .  .N 

-  Let  Zw  be  the  vector  constructed  from  Zy,{t). 

-  The  set  Sm  can  be  written  as: 

Sn  =  {G\z^  =  U5,  <  a  >=  0,  i  =  1, . . . M,  u;  G  W} 

U  is  the  Toeplits  matrix  constructed  from  u  and  p*’s  are  the  annihi- 
lators  of  A. 

—  Eliminating  A 

Sv  =  {G I  z„^Vz„  <  7,  w  €  W} 

V  can  be  computed  from  the  resulting  QP. 

•  Let  Div(7)  =  diam(SA'). 

•  Estimate  7iv  as  follows: 

min  7 

log(iV) 

DaK7)<(C'i7  +  C'2M)— ^ 

•  Estimates  capture  tradeoff  between  parametric  and  non  parametric  er¬ 


rors. 


Example 


•  Process: 

^(*')  =  Tra5A  +  ^W’ 

•  The  unmodeled:  200 

A{A)  =  E  S{k)\'‘ 

fc=0 

/200 

+i)j  PTi 

where  5i{k)  =  (A  -  a)v{k)  and  v,  a  vector  of  length  200,  was  selected 
using  a  random  number  generator. 

•  Noise:  iV(0,  .3) 

•  Input:  High  order  chirp. 


Magnitude  Magnitude 


Example 


Comparisons: 


Process  Ti 


Algorithm 

d 

%  Error 

Actual  Error 

Estimated-Error 

- - — - ^ — 

unmodeled- 

MUigH 

.97 

3 

.03 

HHBKI 

.67 

33 

.33 

i.y 

■i^m 

40 

.4 

2.2 

M| 

Algorithm 

9 

Actual  Error 

unmodeled 

MUD 

.99 

.01 

.06 

1.01 

LS 

.64 

36 

.36 

.0002 

1.9o 

WLS 

.58 

42 

.42 

.0001 

2.3 

Estimating  the  unmodeled: 


magnitude  magnitude 


frequency 


Robust  Control 


•  Standard  robustness  results  extend  for  perturbations  in  %. 

•  Stability  if  and  only  if  for  all  9  G  [0, 27r)  and  A  G  A 

(l  +  (A.  A,)(,^)) 

•  Idea  is  to  compute  the  range  of  A  at  each  frequency: 


•  Through  the  singular  value  decomposition,  we  can  find  a  continuous 

functioning  such  that 

5(e)  =  |vK(e)Q);||r)  ?||2<l} 

.  Stability  if  and  only  if  for  all  9  e  (0, 2™)  and  Ijti)  ?1||2  <  1 

(l  +  h 

•  Above  is  a  standard  one-rank  problem.  Both  analysis  and  synthesis  are 
equivalent  to  convex  problems. 

•  A  theory  for  the  general  robustness  problem  (including  parametric  un- 
certaintv)  has  been  developed  with  such  error  bounds. 


Conclusions 


•  A  new  paradigm  for  system  identification 

—  Bridges  the  gap  between  stochastic  and  deterministic  formulations. 
—  Under  modeling  is  explicit  in  the  formulation. 

—  Based  on  sample-path  analysis. 

-  Captures  tradeoffs  in  error  estimates. 

•  Formulation  compatible  with  robust  control 

-  Derive  error  bounds  in  terms  of  H  •  \\n- 

-  Derive  robustness  results  for  this  norm. 

-  Exploit  the  Hilbert  space  structure. 

•  Future  directions. 
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CONTROLLER  DESIGN  AND  VALIDATION 


m- 


1)  Identification  of  the  dynamic  model(s) 

2)  Performance  and  robustness  specifications 

3)  Compatible  controller  design  method 

4)  Controller  implementation 

5)  Real-time  controller  validation 
(and  on  site  re-tuning) 

6)  Controller  maintenance  (same  as  5) 


Comparison  of 

‘achieved”  and  “desired”  performances 


Iji- 


A  useful  interpretation 


Check  to  what  extent  the  model  used  for 
design  allows  achievement  of : 

-  desired  nominal  performance 

-  desired  robustness  specs,  (sensitivity  fct.) 


r.  ■  J^.  ■vn..A’2?-;r/ii.<  ji.' .-..  ■ 


REAL-TIME  CONTROLLER  VALIDATION 


Time  domain  tests 
( too  inaccurate  in  some  cases  ) 


Adaptation 

Algorithm 


If  the  results  are  not  satisfactory  ; 


I?':-  . . 


Plant  model  identification  in  closed  loop 

+ 

Controller  redesign 


CLOSED  LOOP  PLANT  IDENTIFICATION 


Why  ? 


Iterative  identification  and  controller  redesign 


Cannot  be  dissociated  from  the 
controller  and  robustness  issues 


DUAL  PROBLEM 

CONTROLLER  REDUCTION 


Introduction 

Background  in  robust  digital  control 

Identification  in  Closed  loop 

-  Some  facts 

-  Closed  loop  output  error  (CLOE)  algorithms 

Validation  of  models  identified  in  closed  loop 

Iterative  identification  in  closed  loop  and  controller 
re-design 

Experimental  results 

Controller  reduction  by  identification  in  closed  loop 
Experimental  results 

Identification  in  closed  loop  of  nonlinear  plants 
Conclusions  and  Perspectives 
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Background  in 
Robust  Digital  Control 


The  R-S-T  Digital  Controller 


Plant  Model : 


:^‘7777.'7",vv  *  7;:-  • 

'/*\c  A  ‘'  ^  7  7^  ;•.  '' 


G(q-i)  =  ^ 
A(q 

A(q-i)  =  1  +  ai 
B(q-i)  =  bi  q-i 


q-*^  B(q-^) 

A(q-l) 

1  +  ai  q'^  +. .  .+an^  q'”^ 
biq-i  +...+bnBq'"® 


R-S-T  controller : 


S(q-i)  u(t)  =  T(q-i)  r(t)  -  R(q-i)  y(t) 


Closed  loop  poles  : 


P  =  AS  -t-  z-^BR 


THE  SENSITIVITY  FUNCTIONS 


ROBUSTNESS  MARGINS 


i?;  r,  V'sl 


Modulus  Margin 


AM  =  |1+Hol(z  ^)[niin  —  (|^yp(^  ^)linax)  ”  (||Syp||o®) 


Delay  Margin  : 


At  =  mini  ^ — 

®CR 


Typical  values  : 


AM  >  0.5  (-6dB),  At  >  Tg 


AVP  »  W‘ 


ROBUST  STABILITY 


Im  G 


|5wl 

\ 


(0  =  11 


ReG 


G  (e-i“>) 


uncertainty 

disk 


(0  =  0 


.V"'’ ^  s'  SV.S-  ^  . 

L  -  ^  ^ ,.vr-:<I-  ■■  ^  ..•>  A-v’s-,’  ^  r-  r4>sv^\r  \-  ^  s. 


‘’' ‘ . 

'jv. 


h'4 

f  ?  Ir  \  r' 

1  ’  : 


t.-.vvja 


Family  of  Plant  Models  :  G'  e  F(G,5,Wx) 

G  -  nominal  model ;  ||5(z‘^)||„o  <  1 
Wx(z-i)  -  size  (and  t5^e )  of  uncertainity 


Robust  Stability  Condition  : 


I-.  t 


a  related  sensitivity  v  /  ^  uncertainity 

function  ^  A 


||S,WX  <  1 

^  defines  an  upper  template 

/for  the  modulus  of  the 

jSxl  <  iWxj-T  sensitivity  function 


defines  the  size  of  the 
tolerated  uncertainty 


There  are  also  lower  templates 
(because  of  the  relationship  between  various  sensitivity  fct) 


\  ‘  J'’  "S  'V  t  'V*  ‘ ' 


Templates  for  the  Sensitivity  Functions 


Output  Sensitivity  Function 


nominal 

perform, 


input  Sensitivity  BTinctiori 


|t-ll#f  ROBUST  CONTROLLER  DESIGN 


^  f S I 


Combining  nominal  performance  design 
with  shaping  of  the  sensitivity  functions 


■» 


.sv|r'^§S’%SV'  '"'  @ 


fev 


‘j5|.^fj|^;'%vrV>J'' ' 


-'mb///  •■'^‘‘^'•=" . 


i:*i-'«Sl>;%  j,^,  -1  ’,' 


Pole  placement  with  sensitivity  function  shaping 
using  convex  optimization 

uses  convex  parametrization  w.r.  to 
auxiliary  poles  and  controller  parameters. 


wmi 


"’•fiii©  w,  Aiuf©MAVM(C^  cm:) = jj,iit^wt©i!®  i 


Identification  in  Closed  Loop 


Identification  in  Closed  Loop 


The  Flexible  Transmission 


"open  loop"  mode]  "closed  loop"  model 


0%  load 


fs  s  20  Hz 


- ■  "  I .  I - 1 - 1 - - - 1 - 1 - 1 - 1 - l) 


0.05  0.10  0.15  0.20  0.25  0.30  0.35  0.40  0.45  0.50 


I 
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Benefits  of  Identification  in  Closed  Loop 


The  Flexible  Transmission 


open  Ic 

7.20r^HL 


X  cl.  loop  syst.  pole 
%  computed  pole 


6.80- 


6.60- 


6.40 


6.20 


real  system 


6  A  contrc 

5.90-^ 


IDENTIFICATION  IN  CLOSED  LOOP 


BMv  ‘f  ;s 


OPEN  LOOP  INTERPRETATION 


Obj. ;  Development  of  Algorithms 

_ _ _  .  -  ,...■  ..^  :  ■  ■"  .  /  ■  ■  „•  ^ ^ -  .  ■  -r  .  -  ,  ,  - - > 

take  advantage  of  the  “improved”  input  spectrum 
are  insensitive  to  noise  in  closed  loop  operation 
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CLOSED  LOOP  PLANT  IDENTIFICATION 


Slila 


•wV  V 


-^yi- 

cA  V 


EO  data  filters  :  S/P  or  AS/P  or  .... 
(may  be  related  to  the  control  objective) 


-  biased  estimates 

-  require  (theoretically)  time  varying  filters 

-  FOL  can  be  seen  as  approx,  of  CLOE  alg. 

-  are  used  in  standard  indirect  adaptive  control 
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CLOSED  LOOP  OUTPUT  ERROR 
ALGORITHMS 


e(t+i)  =  e(t)  +  F(t)0(t)ecL(t+i) 

<D(t)  =  (l)(t)  =  [-y{t) , . .  ,a(t) , . (CLOE) 


m)  =  Uit) 


(F-CLOE) 


Unbiased  parameter  estimation  (asympt.) 

Mild  sufficient  positive  real  condition  for 
convergence  (  S/P  or  P/P ) 

Insensitive  to  plant  disturbances 


Objective  of  the 

Identification  in  Closed  Loop 

^  . . . . 

( Identification  for  Control) 


Find  the  “plant  model”  which  minimizes  the 
discrepancy  between  the  real  closed  loop 
system  and  the  “simulated”  closed  loop  system 


W 


CLOE  Algorithms 

(Closed  Loop  Output  Error) 
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IDENTIFICATION  IN  CLOSED  LOOP 


Excitation  on  the  reference 

w. 


Excitation  added  to  the  controller  output 


w 
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Closed  loop  system:  || 

(1)  y(t+l)  = -A*y(t)  +  B*u(t)  =  eVt)  I 

0  —  ^1>**5  Bhb]  * 

(pT(t)  =  [-y(t),..,-y(t-nA+l),  u(t),..,  u(t-nB+l)]  I 

u(t)  =  -§y(t)  +  ru  ® 

^  I ' 

Closed  loop  adjustable  predictor:  * 

(2’)  y°(t+l)  =  -A*(t)y(t)  +  B*(t)u(t)  =  e^(t)(l)(t)  I 

-T  * 

(2")  y(t+l)  =  -A*(t+l)y(t)  +  B*(t+l)u(t)  =0  (t+l)(t)(t)  | 
0  BabJ 

■i 

(|)T(t)  =  [.y(t),..,-y(t-nA+l),  u(t),..,  u(t-nB+l)]  _ 

u(t)  =  -^y(t)  +  fu  I 

Closed  loop  predictioa(  output )  error  I 

ecL(t+l)  =  y(t+l)  -  }^(t+l)  a  priori  I 

£cL(t+l)  =  y(t+l)  -  y(t+l)  a  posteriori  I 
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CLOE 


Adjustable  Predictor ; 
yO(t+l)  =  e’'(t)(t)(t) 

y(t+l)  =  eVl)<l)(t) 

0\t)  =  [ai(t),..,  a„A(t),  bi(t),..,  b„B(t)] 

(j)^(t)  =  [-y(t),..,-y(t-nA+l),  u(t),..,  u(t-nB+l)] 


The  Algorithm 

0(t+l)  =  0(t)  +  F(t)(|)(t)£cL(t+l) 
F(t+l)-i  =  Xi(t)F(t)  +  ;t2(t)(t)(t)(t)^(t) 
0  <  Xi(t)  <  1  ;  0  <  A.2(t)  <  2 

ecL(t+l)  =  - 

1  +  (l)^(t)F(t)(t)(t) 

ecL°(t+l)  =y(t+l)  -  0^(t)(l)(t) 


CLOE  Analysis  J 

( the  “plant  model”  is  in  the  “model  set”  ) 
EcL(t+l)  =  |■[0  -e(t+l)7(t)(t)  +  ^w(t+l) 


Dete 

limt-,„eci 

rmin 

Xt+1 

- - - - - - \ 

istic  environment  (w=0) 

)  =  limt-4oo8cL^(t+l)  =  0 

<  00  ;  Vt 

..J 

V _ _ _  _ _ _ _ _ _ 

Stochastic  Environmei 

Prob(limt^„0(t)  €  Dc 

where  :  Dc  =  (0  :  <|)^(t,0) 

\ _ _ _ _ _ _ _ _ 

It  (w  0  ) 

:i=l 

_0  -  0]  =  0} 

J 

If: 


where  :  maxtX.2  <  A,  <2 
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F  -  CLOE  and  AF-CLOE 


(Adaptive)  Filtered  Cloded  Loop  Output  Error 
ecL(t+l)  =  S'  |[0  -0(t+l)r <t>(t)  =  S [0  -0(t+l)] 


P  S 


F-CLOE: 


AF-CLOE: 


P  =  AS  +  BR;  <t)f(t)  =  I  <|)(t) 


P(t,q-l)  =  A(t,q-l)S  +  B(t,q-l)R  WO 


The  Algorithm 


^  S 
P(t,q-l) 


(t)(t) 


e(t+l)  =  0(t)  +  F(t)(l)f(t)ecL(t+l) 
F(t+l)-i  =  Xi(t)F(t)  +  X2(t)<l)f(0<l)f^(0 

0  <  ^i(t)  <  1 ;  0  <  ^2(0  <  2 

-  /..IX  ecL°(t+l)  /„„ 


£cL(t+l)  = 


1  +  (t)f^(t)F(t)(t)f(t) 


(  approx. ) 


ecL°(t+l)  =y(t+l)  -  0  (0<t>(0 
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F  -  CLOE  (AF-CLOE)  analysis  | 

Deterministic  Stability  Condition : 

Stochastic  Convergence  Condition  (O.D.E.): 


maxtX2  <X  <2 
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Asymptotic  BIAS  -  Frequency  Distribution 

i  (“plant  model”  not  necessarily  in  the  “model  set”) 

I 

I 

!  CLOE/F-CLOE/AF-CLOE  algorithms 

j 

- ^ - - s 

i 

!  0  =  argmine  ■  lSypp[|G  -  Gp  |Sypp(t)r„((o) 

J  -K 

+  (t)w(co)]dco 

_ _ _ y 

(excitation  added  to  the  controller  output) 

excitation  on  the  reference  ; 

Syp^(|)r„(W)  — >  |Turp<|)r(®)  ;  =  TA/P 

Syp  =  Sensitivity  of  the  closed  loop  predictor 

-  noise  does  not  affect  parameter  estimation  (as.) 

-  precision  of  estimated  model  enhanced  in  the 
critical  frequency  regions  for  design 
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Validation  of  Models  Identified 
in  Closed  Loop 


Controller  dependent  validation  ! 

V _ — - - - ^ 

-  Statistical  Validation 


-  Pole  Closeness  Validation 

1)  Identify  the  closed  loop  poles 

( identification  of  the  closed  loop  system) 

2)  Compute  the  closed  loop  poles  based  on 
the  identified  model 

3)  Compare  (visually  and  using  v-gap) 


-  Time  Domain  Validation 

1)  Compare  real  time  results  and  simulation 
results 


Identification  in  Closed  Loop  J 
STATISTICAL  MODEL  VALIDATION 


crosscorrelation. 


N  =  256  -►|RN(i)|  ^0.136 
'  |RN(i)|<0.15  ^ 


Controller  dependent  validation ! 


Identification  in  Closed  Loop 


j  MODEL  VALIDATION  by  POLE  CLOSENESS 

1  1)  Identification  of  achieved  poles  and/or 

1  sensitivity  functions 

by  identification  of  the  closed  loop 


Adaptation 

Algorithm 


I  i  =  estimation  of  achieved  closed  loop  poles 

"  The  same  signals  are  used  for  the  the  identification  of  the  “plant 
\  moder*  in  closed  loop  and  for  the  identification  of  the  “closed  loop” 

'J 

I  2)  Computation  of  the  closed  loop  poles  and/or  the 
'  sensitivity  functions  based  on  the  identified  model 

\  3)  Comparison  (visual  or  using  v  -gap) 
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Iterative  Identification  in  Closed  Loop 
and  Controller  Re-Design 


Step  1  :  Identification  in  Closed  Loop 

-  Keep  controller  constant 

-  Identify  a  new  model  such  that:  8cL 

-  Validate  the  new  model 

Step  2  :  Controller  Re  -  Design 

-  Compute  a  new  controller  such  that 
Repeat  1,  2,  1,  2  ,.••• 

< _ _ _ _ _ _ _ _ _ _ _ _ 

Rem.:  the  first  iteration  gives  the  most  significant 
improvement 
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f 


How  to  use  OL  and  CL  Identification 


i _ 1 _ 4_ 

COMPARATIVE 
CLOSED  LOOP  VALIDATION 

i  ~ 

BEST  PLANT  MODEL 

t  ~ 

CLOSED  LOOP  BASED 
CONTROLLER  (CLBC) 
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Experimental  Results: 
Identification  in  Closed  Loop 
and  Controller  Re-design 


Identification  in  Closed  Loop 

+ 

Controller  Re-design 


The  Flexible  Transmission 


or 
axis 


DC 

motor 


axis 

josition 


Position 

sensor 


Controller 


D 

A 

PC 

A 

D 

C 

C 

adaptech 


WinPIM 


;mod^l,e  ideyitifie 
•en  boucle  ouverte 


•’inodiie  ide^tifi^ 
•  en  boucle  femnee 


0  "o'.05'’o.lO  0.15  0.20  0.25  0.30  0.35  0.40  0.45  0.50 

R^ponses  Fr6quentielles _ _ _ 

Frequency  response  of  the  flexible  transmission 
(identified  models) 


ADAPTECH 


WinREG 


S±m_BO  :  r^ponse  en  simulation 

avec  le  module  identifi^ 


en  Boucle  Ouvearte 


reponse  en  temps  reel 


de  la  boucle  fermee 


45  50 


55  60  65  70  75  80  85  S 


Reponses 


Boucle 


Fermee 


Step  responbse  of  the  closed  loop  system 
(model  identified  in  open  loop(BO).) 


ADAPTECH 


WinREG 


Poles  of  the  Cloed  Loop 
(model  identified  in  closed  loop  (B.F.)) 


ADAPTECH 


WinREG 


R4el  :  r4ponse  en  temps  ar^el  avec 
le  r4gxxlate\ar  lDas4  snx  le 
module  icientifx^  en  BF 


Sim\i  :  areponse  en  simulation 


Reponses 


45  50  55  60  65  70  75  80  85  90 

3  Boucle  Ferm^e 


Step  response  of  the  closed  loop  system, 
(controller  based  on  the  model  identified  in  closed  lop) 


CONTROLLER  REDUCTION 

USING 

IDENTIFICATION  IN 
CLOSED  LOOP 
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CONTROLLER  REDUCTION  J 

-  It  is  an  important  issue  in  many  applications 

-  Several  approaches  can  be  considered 

-  The  most  important :  controller  reduction  should 
be  done  with  the  aim  to  preserve  the  closed  loop 
properties 


Identification  in  closed  loop  offers  two  nseful 
approaches  for  controller  reduction  : 

1)  (indirect)  Identify  a  reduced  order  model  in 
closed  loop  operation 

(will  capture  the  essentials  characteristics  of  the 
model  in  the  critical  frequency  regions  for  design) 

2) (direct)  Reduced  order  controller  identification  in 
closed  loop 

(will  preserve  the  closed  loop  characteristics  in  the 
critical  frequency  regions) 
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-  Algorithms  similar  to  those  used  for  plant  model 
identification  in  closed  loop 

-  The  algorithms  will  try  to  minimize  the 
discrepancy  between  the  two  closed  loops 
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Asymptotic  BIAS  -  Frequency  Distribution  |  . 

I 

I 

■ 

m 


0  =  [fO)f  l>-">fng3l32)"->®ns] 


-  noise  does  not  affect  parameter  estimation  (as.) 

-  precision  of  estimated  model  enhanced  in  the 
critical  frequency  regions  for  design 

-  precision  can  be  further  “tuned”  by  the  choice  of  r(t) 


K  =  ^  (nominal  controller) 
g  =  R  (estimated  reduced  order  controller) 

s 
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Evaluation  of  the  reduced  order  controllers 


I 


-  Comparison  of  the  sensitivity  functions 

-  Comparison  of  the  controller  frequency  characteristics 

Rem.:  The  v-gap  seems  very  useful 
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Use  of  Real  Data  for  Controller  Reduction 


true  closed  loop  system 


nm 


R/S 

-roi 

i 

■  Y 

PLANT 

§ 


R/S 


u 


reduced  order 


q-dB/A 


controller  plant  model 

identified  in 
closed  loop 


P.A.A. 


ecL 


Validation  of  the  estimated  reduced  controller  using 
similar  tests  as  for  plant  model  validation  in  closed 
loop 
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Experimental  results : 
Controller  Reduction  using 
Identification  in  Closed  Loop 


ACTIVE  SUSPENSION 


primary  acceleration  (disturbance) 


main 


chamber 


elastomere  cone 


piston 


residual  acceleration 


'm&m&Z 


nertia  chamber 


Active  suspension  identified  model 


f1  =  31.8 
f2  =  153.i 
f3  =  261.8 

J6  Hz,  xi  = 
1  Hz,  xi  = 
51  Hz,  xi  = 

0.183 

0.071 

0.015 

fs 

=  800  Hz 

g] 


CONTROL  OF  THE  ACTIVE  SUSPENSION 

I 

Model : 

j 

nA  =  6,  nB  =  8  ,  d=  0  ;  (order  of  the  system  :  8) 

Fixed  part  of  the  controller  : 

R=  R’Hr  Hr  =  1  +  q-i  (opening  the  loop  at  0.5fs) 

'  Design  ;  Pole  placement  with  sensitivity  functions 
shaping  (see  templates  next  viewgraph) 

Resulting  controller  (nominal) :  nR  =  11,  nS  =  13 

,  Theoretical  dimension  of  pole  placement  controller : 

nR=5,  nS=  7 
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CONTROL  OF  THE  ACTIVE  SUSPENSION 

- ^  ^ 


Syp  -  Output  sensivity  function 


Sup  -  Input  sensivity  function 
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Dual  Viimicombe  stability  criterion  : 


v-gap 


5(Kn,Ki)  <  b(Kn) 

\  stability  margin 


Controller 

5(Kn,Ki) 

5(Sypn,Sypi) 

6(Supn,Supi] 

b(K) 

Kn 

nR=11,nS=13 

0 

0 

0 

0.079 

K1 

nR=7,nS=9 

0.039 

0.122 

0.041 

0.069 

K2 

nR=5,nS=7 

0.187 

0.119 

0.135 

0.072 

Controller  frequency  response 


0.25 

f/fs 
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-  Applicable  to  nonlinear  plant  models  which  are 
nonlinear  in  the  parameters 

-  P.A.A.  with  a  similar  structure  as  for  the  linear  case 


-  O  contains  computable  gradient  expressions 
(functions  of  u  and  ^y) 

-  The  noise  problems  are  for  the  moment  an  open 
issue 


Identification  in  closed  loop  can  provide  better 
plant  models  for  design 

-  Identification  in  closed  loop  can  not  be  dissociated 
from  the  controller  and  robustness  issues 

-  Specific  closed  loop  identification  algorithms  with 
convenient  properties  have  been  developped 

-  Identification  in  closed  loop  has  started  to  make  its 
way  in  practice 

-  Algorithms  for  identification  in  closed  loop  have 
been  successfully  applied  for  controller  reduction 

-  Algorithms  for  identification  of  nonlinear  plant 
models  in  closed  loop  operation  have  been 
developped 

-  Comparative  evaluation  of  the  controller  reduction 
techniques  (to  be  done) 

-  The  nonhnear  case  raise  numerous  and  interesting 
problems  (to  be  explored) 
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Vision  Based  Navigation 

and 

Control  of  Robotic  Systems 


Joao  Sentieiro  and  Jos6  Santos-Victor 
Institute  Superior  T6cnico 
Institute  de  Sistemas  e  Robotica 
Lisbon,  PORTUGAL 


VisLab  -  Computer  Vision  Lab 


Vision  Based  Control:  First  Steps  ^ 

•  Shirai  and  Inoue  (1973) :  visual  feedback  for 
precise  motion  control  of  a  manipulator. 

•  Hill  and  Park  (1979)  :  “continuous  use”  of 
image  information;  term  Visual  Servoing  first 
used 

•  Weiss,  (1980) :  dynamic  visual  control  of 

robots,  stabilizing  initial  concepts.  ^ 

VisLab  -  Computer  Vision  Lab 


Initial  Difficulties 


Limited  Computing  Power 

Expensive  and  complex  image  digitizers  and 

processors 

Slow  sampling  frequencies  and  dynamics 


k 


VisLab  -  Computer  Vision  Lab 


Why  vision  based  control  of  robots 

Many  robots  used  in  (constrained  /changed) 

industrial  environments 

Need  for  dealing  with  more  general 

environments 

Vision  allows  non-contact  measurements 
(perception)  of  the  environment  (as  humans) 


VisLab  -  Computer  Vision  Lab 


Challenges  for  Control  Science 

•  Noisy  image  measurements 
j  •  time  consuming  (still!)  image  processing 
I  •  large  amount  of  data  to  process 

•  non-linearites  of  camera-lens  system 

•  difficulty  of  inferring  3D  information  from  2D 
images 
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Challenges  for  Control  Science 


•  Large  delays  in  the  feedback  loop 

•  Unreliable  sensor  data 

^  •  Inadequately  modeled  or  difficult  to  model 

plants  or  systems 

’  •  Poorly  conditioned  transformations 


^ _ _ y 
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Image-based  look  and  move  structure 

World  Coordinates 


k 


fe 


Image  coordinates 
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Position-Based  Visual  Servoing 

World  Coordinates 


Cartesian 
control  law 


Pose  Estim. 
Kinem. 


Delays 


Image 

Processing 


Image  coordinates 
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Talk  Outline  (example  cases) 

Visual  Navigation:  Learning  from  bees 

-  Problem 

-  Motivation 

Visual  Docking 

Remote  control  of  Cellular  Robots 
Binocular  Tracking  with  a  robot  Head 

Conclusions 
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Peripheral  flow 


Mi  =— [jCaP0sin0+p0cos0-7i,]+(l+*e)® 


M;,=— [-;c,p0sin0+p0cos0+rii]-(i+x*)0  No  rotatioii 


e  =  Tj—- 


1  1 


Z,  Zi 


Flow  difference 


■■u,+u„  =  L  -x„p0sm0f^-^l+p0cos0f^+^V24  +  xf)0 

\Z/,  Zij 
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Visual  Navigation 
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Ego-Docking 


Visual  Docking 


Eco-Docking 


Control :  Heading 
Speed 


i 
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What  is  the  relative  caiiiera  motion  and  scene  structure  ?? 


**^y>  i" 


>  V'-  ¥<>>  i ^  J 
i  v- - 
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WmM 


Motion  Field 

Rigid  Body  Motion  :  T,  W 
Perspective  Projection 


Optical  Flow 


r  -r 


Planar  Surfaces 


Z(X,y)  =  Zo+Y,^  +  Y/ 


¥ 


—  T 


Z(x,y)  = 


l-Y,^//,-Yv}'//v 


Motion  Model 
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d  .I  /  ^  .y-T 

nt  .+  •*?  /*  V# 


Global  Parametric  Model 

«(^.y) = Ko  +  Jc  +  «,y + u^xy + «„x* 
v(jc,y) = V,  +  v,x  +  v,y + v^xy + v„y* 

Affine  Model 

u{x,y)  =  u^+u^x  +  u^y 
v(x,y)  =  v„+v,x  +  v^y 


Parameters 


“•-41-”’] 


Tz+y.T. 


V  =45il-o1 

•  /.[z,  •] 

T,+y,T, 


,.=lf_LZL,„. 


/A  z. 
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Robot-Eye  Coordination 


m=M  m' 


Ax'  m,,  m,,  ffiijTX 

xy  =  ttij,  "Id  I  ^ 

.  ^  J  L”*>.  1 


'X  7  1  0  0  0  -x'X  -x’y  -x' 
0  0  0  X71  -yx  -yy  -y 


Image  plane 


.1^ 


Image  and  plane  coordinates 
of  4  points  (minimum)  required 


Ground  plane 
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Control  Strategy 


5^ 

•Decoupled  angular  and  linear 
velocities  dynamic  systems 

•Separate  Heading  and  speed 
control  using  the  angular  error 
and  distance  to  target  point 

^ - 


Goal  Point 
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Control  Problem 


structure 


Camera  unknown  Non-Linear 

mapping  -  M 

•Measure  angular  error  between  robot  heading  and  direction  to  target 
•Generate  differential  voltages  to  control  the  robot  heading  direction 
•Similar  loop  can  be  used  to  control  the  linear  speed  as  a  function  of  the  distance  to  goal 
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Robot  head  pose 
Target  pose 
Image  features 


Head/Object  Motion 

5f  =  Jg5q  +  JpSp 

Kinematic  controller 

6q  =  -J~'5f  +  J~'jp  5p 

p  =  K(q,f)  Forward  kinem. 
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Introduces  a  linkage  between  the  camera  and  target  poses 


/  «  K{q,  /“) 


This  constraint  can  be  used  for  simplify  the  system  kinematics 
(Jacobians)  and  dynamics  (decoupling  various  DOFs)  and  one 
can  easily  introduce  motion  models  for  the  target 
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Conclusions 


I? 


‘‘Opportunistic”  use  of  visual  information 
for  control  purposes. 

Challenging  image  geometries 
Ill-conditioned  (perspective)  observations 

Many  issues  of  stability  and  performance 
analysis  remain  to  be  addressed. 
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